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Abstract 

The completion of matrices with missing values under the rank constraint is a non-convex 
optimization problem. A popular convex relaxation is based on minimization of the nuclear 
norm (sum of singular values) of the matrix. For this relaxation, an important question 
is whether the two optimization problems lead to the same solution. This question was 
addressed in the literature mostly in the case of random positions of missing elements and 
random known elements. In this contribution, we analyze the case of structured matrices 
with fixed pattern of missing values, in particular, the case of Hankel and quasi-Hankel 
matrix completion, which appears as a subproblem in the computation of symmetric tensor 
canonical polyadic decomposition. We extend existing results on completion of rank-one real 
Hankel matrices to completion of rank-r complex Hankel and quasi-Hankel matrices. 


1 Introduction 

The problem of completing matrices with missing entries can be traced back to the works of 
Prony in 1795, and has been addressed since in various fields including: compressed sensing 
|CC141 ICR091 IGrollj , system identification and control [DS001IFPST131 ILV09| , graph theory 
jTroOOj . collaborative filtering jCPIOj . information theory |GD05j . chemometrics jBro97] . seis- 
rnics |Hog03[ lKSS13| . estimation problems and sensor networks |CP10j . to cite a few. It also 
appears as a subproblem in the computation of symmetric tensor Canonical Polyadic (CP) 
decompositions [BCMTlfi] . 

1.1 Structured matrix completion 

We are interested in structured matrices of the form 

N 

y(p) = S 0 + Y J PkSk, (1) 

k= 1 

where Sk , k £ {0,..., N } are known linearly independent nxn matrices, and p = [pi,... ,Pn] T ■ 
Without loss of generality, we assume that the matrices Sk £ C nxn are symmetric (a non- 
symmetric matrix completion problem can be always symmetrized). Thus, 5? is an injective 
map C N —> C nxn , called affine matrix structure. The Structured Low-Rank Matrix Completion 
(SLRMC) for complex affine matrix structures is then stated formally as 

p = arg min rank{^(p)}. (2) 

P e C N 
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In the literature on matrix completion, the problem ([ 2 ]) is often formulated in dual form, i.e., as 
rank minimization subject to linear constraints, and is called affine rank minimization [RFP10] . 
In this paper we prefer the formulation (|2j), which seems to be more suitable for completion of 
structured ( e.g., Hankel-like) matrices. 

There are special cases when the solution to SLRMC can be found in polynomial time: 
partially known block-triangular matrices |Woe89] . Hankel/Toeplitz matrices }loh82t 1HR84] . 
block-Hankel matrices FI 191) and some cases of quasi-Hankel matrices [LM09 ], In the general 
case, the low-rank matrix completion is NP-hard (see, e.g., |Fazf)2j ). 

SLRMC can be also considered as an extreme case of Structured Low-rank matrix Approx¬ 
imation (SLRA) problem with missing data |MIJ13j . The latter problem is more relevant in 
practice, where we may have noise in addition to missing data. 

1.2 A convex relaxation 

A popular approach in machine learning [Faz021 IRFPlfl] is to build a relaxation of LRMC, by 
replacing the rank with the nuclear norm (i.e. the sum of singular values): 

p = arg min ||^(p)|L. (3) 

P e C N 

Now © is a convex optimization problem, and a variety of convex optimization methods can 
be used to find its global minimum. A central question is when do the solutions of (J3j) and ([2]) 
coincide (i.e., the conditions on low-rank matrix recovery with the nuclear norm). 

A special case of SLRMC is when each matrix Sk contains only one nonzero element. This 
is the most common unstructured matrix completion: Sq is the known part of the matrix and 
the positions of nonzero elements in Sk indicate the values to be completed. There is a vast 
literature on the subject on low-rank matrix recovery, where mainly the case of unstructured 
matrix completion is treated. Most of the results are proved in a random setting [CR09IIRFPIUI 
IGroll] : the positions of missing entries (non-zero elements of Sk, k G {1,..., N}) are assumed 
to be drawn randomly; often the known entries are also assumed to be random. There are 
also results on the general affine rank minimization (which is equivalent to SLRMC), but the 
linear constraints are also assumed to be drawn randomly according to a certain probability 
distribution. 

Up to the authors’ knowledge, there exists only one result pP15| . which treats the case of 
fixed structure, in a very simple case. 

Theorem 1 ([DP15, Thm. 1] ). Let Sf be a Hankel matrix 



' 1 

A ••• 

X n ' 


y(p) = 

A 

A 2 ••• 

Pi 

5 


_A n 

Pi ■■■ 

Pn-l. 


where A G (—1:1). Then the solution of ([ 3 ]), with p G M" 1 , is unique and coincides with the 
minimal rank (rank-1) completion, which is given by pk = A n+/ \ 


In this paper we extend the results of |DP15) in two directions: (i) to Hankel matrices 
with arbitrary rank, and (ii) to quasi-Hankel matrices, which are particularly interesting in the 
context of symmetric tensor CP decomposition |BCMT1~0] . As in [DP 15 ], we consider cases 
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when the solution to SLRMC is known, and establish conditions on when the solution of ([ 2 ]) 
and ([3]) coincide. 

Our results are more general (for example, applicable both in the real and complex cases), 
and do not depend on the results of |DP15| . However, we believe that our exposition may also 
help to understand the complicated logic of the proof in [ DP15j . 

In Section [2] we introduce the main notation, in particular quasi-Hankel matrices, which are 
generalizations of Hankel matrices. In Section [3] we formally state the considered quasi-Hankel 
matrix completion problem, and summarize some known results (this summary is needed for 
stating the main results of the paper in Section[6]). In Section[4]we recall optimality conditions for 
convex optimization problems, and specialize them to nuclear norm minimization for structured 
matrices. 

Section[5]contains the main lemmas needed for proofs of the results of the paper. The lemmas 
imply that the optimality conditions are satisfied if the column space of completed matrices is 
not too far from a certain simply structured subspace. Finally, in Section [hj we prove the main 
results of the paper, and illustrate them with numerical experiments in Section [7j 

2 Background and main notation 

2.1 Sets of multi-indices 

We denote by N the set of nonnegative integers. First, for a multi-index a = (ai, 02 , • • • , ot m ) G 

N m , the monomial xf^x'f 2 ■ ■ ■ x will be denoted as x a , and its total degree is |a| = f Yhn a £- 
For ordering the multi-indices, we use in this paper an ordering denoted by It is defined 
recusively on the size of vectors a and (3 as: 

ol -< (3 

( |a| < \(3\ 

< or 

[ |ot| = \(3\ and (a 2 , •.., a m ) -<! (/3 2 ,..., fi m ) 

Next, we shall denote by ±( m ’ d ') the set of multi-indices {a G N m : |a| < d}. By A^ m,d ' > we 
denote the set {a G N m : |a| = d}. It is easy to see that 

±{rn,d) _ ^(m,0) u y . . . y _ 

Example 1. Take m = 2 and d = 3. Then 

A ( 2 ’ 3 ) ={( 0 , 0 ), ( 1 , 0 ), ( 0 , 1 ), ( 2 , 0 ), ( 1 , 1 ), ( 0 , 2 ), 

(3,0),(2,1),(1,2),(0,3)}, 

A( 2 ’ 0 ) ={(0, 0)}, A^’ 1 ) = {(1,0), (0,1)}, 

A< 2 ’ 2 > ={(2,0), (1,1), (0,2)}, 

A< 2 ’ 3 ) ={(3,0), (2,1), (1,2), (0,3)}. 

The multi-indices are ordered according to -< in the sets defined above. In fact, when the total 
degrees are the same, terms are sorted only by increasing degrees in the second variable. 

def 

For sets A,B C. N m , we define their Minkowski sum as A + B = {a. + (3 : q:G^4, /3g B}, 

with a shorthand notation 2A d = A + A. It is easy to see that + A^ m,d2 ^ = A^ m,dl+d2 ^ 

and = +d 2 )' 
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For m = 1, we have that A (1,rf ) = {0,..., d} and {0,..., d±} + {0,..., cfo} = {0,..., d\ + cfo}- 
For m = 2, an example is shown in Fig. [I] (the multi-indices are depicted as black dots). 



Figure 1: Minkowski sum: A (2,3 ) + A (2 ' 3 ) = A (2,6 ). 

Finally, for A 6 N m , we define its extensior0as 

A + d = Au(A + {ej) U • • • U {A + {e m }), 

where e^, 1 < k < m, denotes the vector of N m having a 1 in the k -th entry and zeros elsewhere. 
The exterior boundary of A is defined as 

6A = A + \ A. 

For example, for A = we have that 

A + = A (m,d+l) j $ A = A (m ’ d+1) . 


2.2 Quasi-Hankel and quasi-Vandermonde matrices 

Let A be a multi-index set ordered according to -<: 


A = {ai,...,a M }, OkfN 1 


(4) 


and h = {h a }a£V C C be an array of numbers, where 2 A C T> C N m . Then the symmetric 
quasi-Hankel jMPOOj matrix is defined as 


^A(h) * \h ai+a r& 


M,M 




Example 2. For A = A^ 1,d ) = {0,... ,d}, M = d + 1, the quasi-Hankel matrix is the ordinary 
Hankel matrix 

"ho hi • • • hd 


h) = [h k+ i\i ;to = 


hi h 2 1 h 


~ hd h, 


d +1 • • • 


d+1 


h 2 d - 


(5) 


Example 3. In case m > 1, for example A = A^ 2 " 3 -* (as in Fig. [f]) the matrix i(h) has the 
form 


ho.o 

h 1,0 ho.i 

h2.o hi,i ho.2 

h3,0 h2,l hi,2 ho,3 

hl,0 

ho,l 

h2,0 h 1,1 
hi,i ho,2 

h3,0 h 2 ,i hi,2 

h 2 ,i hi,2 ho,3 

h4,0 h3,i h2,2 hi,3 
h.3,1 h2,2 hi ,3 ho,4 

^2,0 
hi,i 
ho ,2 

h3,0 *2,1 
h 2 ,i hi, 2 
hi,2 ho.3 

h.4,0 h3,l h.2,2 
h3,l h 2 ,2 h 1,3 
h 2 ,2 hi,3 ho.4 

h. 5,0 h4,i h3,2 h 2 ,3 
hl,0 h3,2 h 2 ,3 hi,4 
h‘3,2 h 2 ,3 hi ,4 ho ,5 

^3,0 

h 2,1 
hi,2 
ho,3 

hi,o ho,i 

h3,l h.2,2 
h 2 ,2 hi,3 
hi,3 ho, 4 

hs.O h. 4,1 h3,2 
h4,i h3,2 h 2 ,3 
h'3,2 h 2 ,3 h 1,4 

h2,3 hi ,4 ho,5 

h6,0 h5,i h-4,2 h3,3 
h5,l hi ,2 h 3,3 h2,4 
h4,2 h3,3 h2,4 hi ,5 
h3,3 h2,4 hi ,5 ho ,6 


(6) 


Tn the papers [BCMTIO] , [LM09| and others this notation was used for the set of monomials. 
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where we omitted parentheses in subscripts for conciseness. Note that in ([6]) ^ each block corre¬ 
sponds to a subset A0 < k < 6. 

Let A C N m be the ordered multi-index set defined in and z\,...,z r E C m be a set of 
points. Then the quasi-Vandermonde matrix is defined as 


■ • ■ > z r) 


def 


[(*;)“*] 


N,r 

i,j =1 ' 


In particular, for m = 1, A = A (1,( h = {0,..., d}, it is just an ordinary rectangular Vandermonde 
matrix. 

Example 4. For A = A (2 ’ 3 \ r = 3 and Z}. = \^\, k E {1,2,3}, we have that 1A{z\, z 2 , z %) = 


' 1 

Ai 

Xj XiP! p\ 

A? X\pi Ai pi pi 

1 

P'2 

xl X 2 P 2 pI 

A 3 ^lP2 X 2 p 2 P 2 

1 

A 3 p3 

Ai A 3 ^3 pI 

A3 A3 P 3 A3/X3 pi _ 


Finally, we will use the following definition. 


Definition 2. The points z±,... ,z r E C m are called A-independent if 


rankY^\{z \,..., z r ) = r. 

The notion of ^-independence is equivalent to the fact the monomials {x a } a ^ taken on 
the grid of points {zi,..., z r } form a set of if A vectors spanning a linear space of dimension r. 
Hence, these monomials can interpolate any function on this grid. 

If m = 1, the points are always A-independent for any A with r < if A. Although for m > 1 
this is no longer the case, the following remark holds true. 

Remark 3. A-independence is a generic property, i.e., points {zj}} =1 randomly drawn according 
to an absolutely continuous distribution are A-independent if r < if A. This follows from the 
fact that the set of A-independent points is open in the Zariski topology. 


3 Hankel and Quasi-Hankel matrix completion 

In this paper, we consider the following problem. 

Problem 1. Given A = and {h ct } Q , e ^, 

minimize rank , (8) 

h a ,a£2A\A 

where h = {h a } a( z 2A . 

If m = 1, we have A = {0 ,... ,d} (the matrix J4?a{ h) shown in ([5])); in this case, only the 
values ho,..., h^ are known (shown in gray in ([5])) and hd+ 1 ,..., h- 2 d are to be completed. The 
matrix completion for Hankel matrices is entirely solved 11 RS I . and its solution is related to the 
Sylvester’s algorithm for CP decomposition of 2 x • • • x 2 tensors [ IK9 9]. We recall the Hankel 
case in Section [3TT1 

In the general case (m > 1), the upper block-triangular part of the quasi-Hankel matrix 
is known ( e.g., in ([6]) it is shown in gray). As it was shown in HC.\1 Tl( ) . symmetric CP 
decomposition can be reduced to solving Problem[l] The general quasi-Hankel matrix completion 
problem ([8]) can be solved in some cases when the rank is sufficiently small (this is exactly the 
case when the CP tensor decomposition can be solved without considering incomplete quasi- 
Hankel matrices, e.g., using the Catalecticant algorithm [IK991 lQQ13] h We recall these cases 
in Section [3721 
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3.1 Solution of Hankel low-rank completion 

Probably the first complete solution of Problem [I] for m = 1 is contained in fIoh82l Ch. II], 
where the problem of matrix completion is called “singular extension” of Hankel matrices. In 
this section, we will use the theory from | HR84 ], and we will provide a summary of results in a 
simplified form. 

For a finite sequence h d = [ho ... h d ] E C rf+1 , we denote by hrank (hd) the smallest 
number r such that there exists a non-zero vector q = [qo Q\ ■ ■ ■ q r \ T 0 for which 

q 0 h k + ■ ■ ■ + q r hk+ r = 0, Vfc E {0,... , d - r}. (9) 


The value hrank (hd) is equal to the maximal rank of the submatrices of the known triangular 
part of the Hankel matrix ([5]) (or, equivalently to the maximal rank of a Hankel matrix that can 
be constructed only from the elements of the vector hd). The number hrank (hd) is called the 
first characteristic degree of hd [HR84, Def. 5.3, page 81]. We will also call the vector q as the 
characteristic vector^] of hd- Next, we recall the following basic properties of hd and q. 

• For any hd E C d+1 , hrank (hd) < (see 11H8 1 . Prop. 5.4]). 

• If, in addition, hrank (hd) < , then the characteristic vector q is unique up to scaling 

by a constant (see [HR841 page 84]). 

In what follows, we assume that q r 0 (this is a generic case). The treatment of the case q r = 0 
can be found in [ HR84 , page 84]). Then an explicit representation of hd is determined by the 
characteristic polynomial 

r s 

q{ z ) = = qr^2i z ~ Afe)"*, (io) 

j =0 k= 1 

where A^ E C are distinct, u k are the multiplicities of the roots, and r = v\ + ■ ■ ■ + v s . 

Proposition 4 (A special case of |HR84) Thm 8.1]). If q r 0 and all the roots of q(z) are 
simple (v k = 1, s = r), then the sequence hd admits a representation 


h k [ Cj A 

3 = 1 


(ID 


Example 5. For h d = [ 1 A • • • X d ] (see Theorem |i|) ; we have hrank {h d ) = 1 and the 

characteristic vector can be chosen as q = [—A l] T . 

The representation © is called a canonical representation of h d (or the unique canonical 
representation in case hrank (h d ) < ^w)- In the general case (10), there are multiple roots and 
the canonical representation has a more complicated form, as stated by the proposition below. 

Proposition 5 (A special case of [HR841 Thm 8.1]). Let q r 0, Ai = 0 and A k 0, and the 

multiplicities v k of the roots \ k in (10) are such that u\ > 0 and > 1 for k > 2. Then the 

sequence h d has the form 

S V\ — l 

hk = ci t i6(k,l), ( 12 ) 

j =2 i=o 

where Cj(k ) are polynomials of degree [y k — 1) and 5(k,l ) is the Kronecker symbol. 


“In IHR84I . q does not have a specific name and is denoted by p. 
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Remark 6. In the statement of Proposition^ if u\ 

Example 6. In the extreme case q(z ) = q r z r (i.e., 
becomes 

hd = [ci,0 • • • Ci : r-1 


= 0 then the second term in (12) is absent. 


r = vi), the canonical representation (12) 
0 ••• 0] T , (13) 


Apart from the explicit form of the sequence hj, the characteristic polynomial gives a solution 
to the rank minimization problem ([8]) (for m = 1). 

Proposition 7 (A corollary of IHR84 Thm. 5.14]). Let q G C r+1 be a characteristic vector of 
hd with q r A 0, r = hrank (hd). Then it holds that (i) The rank of the minimal rank completion 
of h) in © is r. (ii) A minimum rank completion hd+i, ■ ■ ■, h 2 d is given by the recursive 
continuation, for k > d — r: 


^k+r (qohk 4 “ ' ' ' 4 “ qr—\hk-\-r—l) 

Qr 


(14) 


In addition, the values of hk+ r for k > d — r can be obtained by using the corresponding formula 
of canonical representation, namely © or ( p~2[ ) . (Hi) If r < 
completion is unique. 


d+ 2 
2 


then the minimum rank 


3.2 Solution of quasi-Hankel low-rank completion 

Unlike Hankel matrices, for m > 1 the quasi-Hankel completion problem does not have a closed 
form solution. The simplest case when the solution can be easily found is given by a flat extension 
theorem [LM091 Thm. 1.4]. 

Proposition 8 (Corollary of ILU09 . Thm. 1.4]). Let A = ^ m ' d \ and the values {h a } a ^ be 
given. Moreover, let d' = f [Jj, B and it holds that 

rank{J^%(h)} = rank{J^+(h)} = r. 

Then the unique solution of Problem^ has rank r. 

Proposition [8] induces bounds on rank that are too restrictive. For example, for A = A* 2,3 \ 
d’ = 1 and only rank-one cases can be treated. In what follows, we consider arrays of special 
form, for which Proposition [8] can be extended. 

Let C C N m be arbitrary, z\,... ,z r G C m , c \,..., c r G C be some coefficients, and the array 
of coefficients h = {h a } a£ 2C be defined as 

r 

ha = ^2 CkZ™. (15) 

fc =1 


It is easy to see that the following lemma holds true 


Lemma 9. For h defined as in (15), the quasi-Hankel matrix admits the factorization J#c( h) = 

Vc{z l, z r ) diagjci,... ,c r }(Tc(zi, .. ,z r )) T (16) 


and rank{J^(h)} < r. If, in addition, ck A 0 f or a M k, and z \,... ,z r are C-independent, then 


rank{J^c(h)} = r. 
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Next, we describe an easy improvement of Proposition [8] for arrays of form (15). 


Proposition 10. Let A = k.( rn,d \ d! = \^\, B = A^ m,d ^ (it is easy to see that 2 B C A). 
Assume that the coefficients {h a } a£ ^ have the form (15), where the points zi,...,z r are B- 
independent and c\,...,c r are nonzero. Then, the following hold true (i) The rank of the 
minimal completion in Problem [I] is r. A minimal rank completion is obtained by defining 
{h a }a£2A using the same formula (15) (called canonical completion), (ii) If d is odd, the 
canonical completion is unique. (Hi) If d is even, and z\,...,z r are A^ m,d ~ l ^>-independent, 
then the completion is unique (the canonical completion of Eq. (15) is the only possible). 

Proof. See Appendix |8.1| for a proof. □ 


For example, for the completion of structure shown in ([6]) (m = 2 and d = 3), Proposition [To| 
covers cases up to rank 3. Indeed, let Zk = [^ ], k E {1, 2,3}, we have that if the Vandermonde 
matrix has full rank Q, then the matrix completion is unique and is given by the canonical 
completion. 


Remark 11. If the points z\,...,z r are generic, due to Remark [3| we can replace the B- 
independence and A ( m > d,_1 ) -independence by the bounds r < ( d +rn ) and r < ( d+ " l_1 ) respec¬ 
tively. 


Finally, we note that for generic cases, the rank bound in Remark[lT]can be further improved 
by using results of (OQ13I Theorem 2.4] (since uniqueness of matrix completion is related to 
uniqueness of tensor decomposition). For our purposes, however, Proposition[lO]will be sufficient. 


4 Optimality conditions of nuclear norm minimization 

4.1 Optimality conditions in convex optimization 

We recall some definitions from the field of convex optimization (HULOll Chapter D], For a 
convex (possibly non-differentiable) function / : M. N —> M, the subdifferential (HULOll Def. 
1.2.1] of / is defined as the set df(x ) C M. N 

df{x) = f {d : f(y) - f(x) > (d,y - x) Vy <E 1^}. 

In particular, if / is differentiable at a point x, then the subgradient has only one element: the 
usual gradient, i.e., df(x) = {V f{x)}. 

For the unconstrained convex optimization problem 


min f(x 


(17) 


we recall the first-order optimality condition, which is necessary and sufficient in this case. 


Lemma 12 (First-order optimality, jHIJLOll Thm. 
© if and only if 


0 E df(x). 


2 .2.1]). A point x is a minimum point of 


(18) 


In particular, for differentiable functions, the condition of Lemma 12 reduces to V/(*) = 0. 
Next, we use a simple uniqueness condition for the minimize!'. 















Lemma 13 (Sufficient condition of uniqueness). A point x is the unique minimizer of © if 

0 E int (df(x)), (19) 

where int (•) denotes the interior of a set. 

Proof. In this case for any y ^ x there exists 6 > 0 such that 5 ■ (y — x) E df(x). By definition 
of df(x), we have that f(y) — f(x) > S\\y — x\\\ >0. □ 


4.2 Subdifferential of the nuclear norm 


First, we recall the form of the subdifferential of the nuclear norm of a matrix. Let X E R niXTl2 
be a matrix of rank r, and let X = I/Sy T be an SVD of X, where U E M niXr , V E M n2Xr 
and 51 E M rxr is a diagonal matrix of nonzero singular values. Next, let U± E M niX ( ni-r ) ; 
V± E M ?l2X ( ri2 r ) be the bases of the left and right nullspaces of X, respectively. Then, according 
to [Wat92 : . p. 41], the subdifferential of the nuclear norm at X is equal to <9||X||* = 

{B + U l WV2 : W E \\W\\ 2 < 1}, 


where B is defined as 

and 


B d = UV T , 

2 is the spectral norm (largest singular value). 


( 20 ) 


Remark 14. When the SVD of X is not unique, then the matrix B from (20) is still defined 
uniquely. In fact, if X has multiple singular values, its matrix of left (resp. right) singular 
vectors is of the form UQ (resp. VQ), where 0 is an orthonormal matrix which commutes 
with S. 


Next we consider a real nuclear norm minimization problem 

P = arg min / (p), f(p) = f ||^(p)||*, (21) 

pe R N 

where 5? is defined in ([Tj) . By an analogue of chain rule jHULOll Thm. 4.2.1], we immediately 
have that 

df( P ) = {[{s u h) f ••• (s n ,h) f } t -. 

H ed\\X\\*,X = y(p)}, (22) 

where (•, -) F denotes the Frobenius inner product. In the following section, we will give a more 
compact form or subdifferentials and optimality conditions. 


4.3 Real nuclear norm minimization 

First, we assume that X is symmetric and n\ = n 2 = n, then we have that U±U)[ = V±V^ = Q , 
where Q is the projector on the nullspace of X. Then the subdifferential can be rewritten as 

d\\X\\* = {B + QMQ :\\M\\ 2 <1}, (23) 

where B is defined as in ( po] ). We note that Q = I — P, where P is the orthogonal projector 
on the column space of X, which can be obtained as 

P d =l f UU T = BB J . (24) 


9 







Next, we define the matrix 


S = f [vec(Si) 


and for a matrix P E 


we define 


c(Sn)] , 

(25) 

P)) E R Nxn \ 

(26) 


where <8> denotes the Kronecker product. Then, by (22), the subgradient of / can be rewritten 
as df(p) = 

|s T vec(B) + */(P) vec(M): ||M || 2 < l} . (27) 

with M E R nxn . Using (27), we can rewrite Lemma 


12 


and Lemma 13 as follows. 


Lemma 15 (First-order optimality conditions for real nuclear norm minimization). A point p* 
is a minimizer of ( |21[ ) if and only if there exists a matrix M E R nxn with ||1V4" || 2 < 1 such that 

t/(P) vec(M) = -S T vec(B). 


where B is as in (20), P is as in (24), and Q = I — P. 


def 


Proof. Follows from (27) and Lemma 12 


(28) 


□ 


Lemma 16 (Sufficient condition of uniqueness). A point p is the unique minimizer of © */ 
there exists M E R nxn with ||jVf|| 2 < 1 such that (28) holds and the matrix £^{P) is full row 
rank (Te., is of rank N). 


Proof. Follows from (27) and Lemma 13 


□ 


4.4 Complex nuclear norm minimization 

Now let y(p) be the matrix structure given in (jT|) , but now Sq E C nxn , Sk E R nxn for k > 1, 
and the parameter is complex pc E C N , i.e., 0 defines a complex-valued map 5? : —> C nxn . 

In order to derive the optimality conditions for the complex-valued case, we construct an 


extended real-valued structure y ex t : R 27V —> R 2nx2n as follows. For p ext = 
PUiPi ^ we define an equivalent (non-symmetric) matrix structure: 


Pn 

Pi 


y e xt{Pext ) 


'^{Pn) ~y{px) 

Pi ) Pn) . 


, where 


(29) 


From jDHOl] . the singular values of df,’ ex t(Pext ) contain two copies of singular values of J^(pc). 
Hence, 

rank{y ext (pext)} = 2rank{^(p c )}, 

and the complex rank minimization problem ([2]) is equivalent to real rank minimization 

(30) 


p ext = arg min rank SA ext ( p ext ). 

Pea:t6lR 2JV 


Moreover, the problem ([3]), is equivalent to the real nuclear norm minimization problem 

Pext = arg min J^ext(Pext)||*- 

Pext eM 2JV 


(31) 


The following proposition shows that we can rewrite the optimality conditions for the problem 


(31) in a convenient form, as in Lemmae 15 and 16 
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Proposition 17 (Complex optimality conditions). Let p c = pn + ipz G C N , y{pc) = t/ c EV^ H 
be an SVD of the symmetric matrix 27 {pc), and define Be := U c V£, Pc = U c Uq, and 
Qc = In ~ Pc- Then it holds that: 

(i) The point pc is a minimizer of (31) if and only if there exists M E C nxn with ||IW ||2 < 1 
that satisfies (28). 

(ii) The point p ext is the unique minimizer of (31) if there exists M E C nxn with |||| 2 < 1 
that satisfies ( [28] ), and &f(P) is full row rank. 

a 

Remark 18. Lemmae \ 15\ \16\, and Proposition \ 1 7| are related to optimality conditions used in 
' DPI 5 7. Indeed, the rank condition on rank {&/(P)} = N is equivalent to QHQ T A 0 for all H 
of the form 

N 

\/H = Y,^PkS k , ApEC^UO}, 

k =1 


Proof. See Appendix [8TT] for a proof. 


which corresponds to IDP1 51 Proposition 2]. Next, the condition (28) can be rewritten as 

(S k ,B + QMQ T ) F = 0, Vfc E {1,..., N}, 

which corresponds to the condition in IDP12A Lemma 2]. The authors of fDP15 i. however, do not 
make explicit connection with general optimality conditions for convex optimization problems, 


presented in Section 4-1 


5 Main lemmas: optimality conditions and simple projectors 

5.1 Quasi-Hankel completion: basis matrices 

First, we put the rank minimization problem J8]) in the (standard) form ([ 2 ]). In order to do this, 
we explicitly write down the matrices S k in (fTl). The constant part has the form 

So = ,yp A (c), 


where c = {c^fa^A is defined as 


def I h a , OL E A, 

Cq ~~ \o, ol E 2A\A. 

Example 7. In the case of A = A (2,3 i (depicted in we have that Sq = J^a(c) = 


ho,o 

hi,o 

^0,1 

^2,0 

h\,l 

ho,2 

^3,0 

^ 2,1 

hi,2 

ho ,3 

hi,o 

h 2,0 

hi,i 

^ 3,0 

ft-2,1 

hi,2 

0 

0 

0 

0 

ho,i 

hi,i 

ho,2 

/l 2 ,l 

hi,2 

ho,3 

0 

0 

0 

0 

hi,o 

h 3,0 

/l 2 ,l 

0 

0 

0 

0 

0 

0 

0 

hi,i 

h 2 ,i 

hi,2 

0 

0 

0 

0 

0 

0 

0 

ho,2 

hi,2 

ho,3 

0 

0 

0 

0 

0 

0 

0 

ho,o 

0 

0 

0 

0 

0 

0 

0 

0 

0 

^2,1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

hi,2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

ho,3 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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In the case of Hankel matrices (A = A (1,d ), see ([5 ]) ), we have that n = 


So = 


ho 

hi 


hd-i 
. hd 


hi 

h'2 


hd 

0 


hd -1 
hd 

0 


h d 

0 


The number of free parameters in ([Tj) is AT = ff(2A) — ff(A) ( i.e 
in (|8]) ). The corresponding set of multi-indices can be represented as 

2A \ A = {/3i,..., /3 tv}> /3i -<■■■-< (3n- 


Next, we define basis arrays e^ 3 c = as 


p /3 dof 



a = (3, 

a A (3, 


and Sk as 

S k := 

Finally, it is easy to see that 

J&(h) = ^(p), 

if p is dehned as pk = h-p k . 

Example 8. For A = A (2,3 \ shown in Q , we have 


0|0 0|0 0 0|0 0 0 01 


ro|o o|o o o|o o o o 

0 

0 0 

0 0 0 

10 0 0 


0 

0 0 

0 0 0 

0 10 0 

0 

0 0 

0 0 0 

0 0 0 0 


0 

0 0 

0 0 0 

10 0 0 

0 

0 0 

10 0 

0 0 0 0 


0 

0 0 

0 1 0 

0 0 0 0 

0 

0 0 

0 0 0 

0 0 0 0 

— 

0 

0 0 

10 0 

0 0 0 0 

0 

0 0 

0 0 0 

0 0 0 0 

5 ^2 

0 

0 0 

0 0 0 

0 0 0 0 

0 

1 0 

0 0 0 

0 0 0 0 


0 

0 1 

0 0 0 

0 0 0 0 

0 

0 0 

0 0 0 

0 0 0 0 


0 

1 0 

0 0 0 

0 0 0 0 

0 

0 0 

0 0 0 

0 0 0 0 


0 

0 0 

0 0 0 

0 0 0 0 

0 

0 0 

0 0 0 

0 0 0 0 


0 

0 0 

0 0 0 

0 0 0 0 

0|0 0|0 0 0|0 0 0 01 





0 

0 0 

0 0 0 

0 0 10 





0 

0 0 

0 0 0 

0 10 0 





0 

0 0 

0 0 1 

0 0 0 0 





0 

0 0 

0 10 

0 0 0 0 





0 

0 0 

10 0 

0 0 0 0 

, . . . 




0 

0 0 

0 0 0 

0 0 0 0 





0 

0 1 

0 0 0 

0 0 0 0 





0 

1 0 

0 0 0 

0 0 0 0 





0 

0 0 

0 0 0 

0 0 0 0 






For Hankel structure, (A = A^ 1,d \ shown in ([5])^, we have that N 
k G {1,... ,N} are defined as 


Si = 


S N = 


r° 

0 

... 0 

0-1 


r° 

0 ■■■ 

0 

0 

0 

• 0 

1 


0 

0 





0 

, ■ ■ • , Sn-i = 




0 

0 




0 



Lo 

1 

0 ■■■ 

6-1 


Lo 

... 0 

1 

r° 

0 

... 0 

0-1 




0 

0 

0 

0 




0 

0 


0 




Lo 

0 

... 0 

1-1 





d+ 1, and So is given by 

(32) 

the number of unknowns 

(33) 


(34) 


d, and the matrices Sk, 


(35) 
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5.2 Simple matrices 

Now we show that optimality conditions hold true for simple matrices. We take A = A.( m,d \ 
and consider the matrix completion problem (|8j), where Sk are defined in Section 5.1 


Lemma 19. Let d' = f |_fj > Po £ C nxn be a matrix of the form 


def 
Jri 1 — 


n o 
0 0 


Pn G C S 


(36) 


where s < Then it holds that (i) For any k G {1 ,...,N}, P 0 T SkPo = 0. (ii) If i 

addition, s < n" := ( rf + m ), where d" := then rank{j2/(Po)} = A. 


Proof. See Appendix [87T| for a proof. 


□ 


Example 9. In the case of Hankel matrices (A = A (1 ' rf ) ) the first bound is s < an d the 

first statement of Lemma 19 is evident from (35). The second bound is s < L^rJ; an< ^ a ^d^t 
lower block of the matrix QqHQq (where H is defined in Remark IS) has the form 


- 0 

0 Api 


<d 




Ap r+ i 

0 




Apr 



Apn-2 

.A Pr Ap r+ 1 

• • • Apat-2 

ApiV_2 

Apiv_i 

Apn-1 

Ap N 


and hence all Apk are present in this block. 


Lemma 19 immediately proves that the SLRMC is solved by nuclear norm minimization in 
very simple cases. 


Corollary 20. In the completion problem Q for the structure ([5]), let the vector h<i be of the 
form ( [l3| ), with r < dfA. Then we have the following. 

(i) A solution of the rank minimization problem is given by hk = 0 for k > d, and coincides 
with the solution of ©• (ii) If, in addition, r < the solution of © is unique. 


and Pq = I s 


Proof, (i) Let Po be as in (J36J) , with s = r 
form Pc = P 0 AP f T By Lemma 
follows from Remark [l8| (ii) Follows from Remark 18 


Then, the matrix Pc in (20) has the 
we have that (Sk,Bc)p = (P 0 T SfcPo, A)p = 0. The rest 

□ 


5.3 Perturbations of simple projectors 

Now we show that the conditions of Proposition[l7]hold true for perturbations of simple matrices. 
First, we need a simple inequality on distance between projectors. 

Lemma 21. Let P = P T = UV T , where U, V G M nxr such that U H U = V H V = I r . Let P 0 
be an orthogonal projector on a subspace of C n , such that rank(Po) = r. Then we have that 

||P - P 0 PP 0 T f F < 21|(P - P 0 )U\\ 2 p = ||P - Poll?,, 

where P = PP H = UU H . 
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Proof. Since B-PqBPJ = B-PqB+PqB-PqBPJ , 


\B-PnBP 


T || 2 


< 


I-P 0 )B\\ 2 F + \\P 0 B(I-Pj'" 2 

2 _ Oil t T T> 


<211(7 -Po)BK = 2\\(I-P n )U 


Finally, 


iP-Pr 


0||F 


= trace{P 2 + P 0 2 — PPq — PqP} 


= 2trace{2(7 - P 0 )P} = 2||(7 - P 0 )P" 2 


F> 


which completes the proof. 


( 37 ) 


n 


Remark 22. For a symmetric matrix F^{p), the matrix Be defined in Proposition 17 is also 
symmetric. 

Next, we prove a perturbation lemma that uses a bound on the distance between projectors. 


Lemma 23. Let S and srf(P) be defined in (25) and (26). Let Pq £ C nxn be a rank-r projector 
matrix such that rank{._6/(Pj)} = N, P 0 T SkPo = 0 for all k £ {1,..., TV}, and let Qq c = I — Pq. 
Then there exists a constant e such that for any B as in Lemma \2l\ satisfying 


||P — Pq\\f < £) 

where P = BB H , it holds that (i) The matrix £/(P) is full row rank, (ii) There exists a matrix 
M £ C nxn , \\M \\ 2 < 1, such that (28) is satisfied. 

Proof, (i) Since £&(P) depends polynomially on the entries of P, and rank^/(Po) = N, the 
rank is preserved in a neighborhood of Pq. Thus rank,e/(P) = N in a neighbourhood of Pq. 
(ii) Let £i > 0 be a number such that for any P, ||P — Po||f < £i we have g/(P) = N 
and 0 < 5q < cr m i n (g/(P)) for some number 6q. (We can choose such e\ by continuity of the 


smallest singular value.) Then, in the £i-neighborhood, a solution of (28) always exists. Next, 
we consider the minimum Frobenius norm solution il7* = A7*(P) of (28), which is given by 


vec 


(Af*(P))) = -i/(P) t S T vec(P) 


= —stf (P)^S T vec(P - P 0 PP C 


0 )i 


where the matrix £&(P)^ is the pseudoinverse of srf (P), and the last equality holds since 
pjs k p 0 = 0 for all k £ {1,..., N}. Then, by Lemma pTJ 


|TV/*11 2 < || vec(M*(P))|| 2 < 


\B — P 0 BPr' ||i? < 


Til- - ll S ll 2 "p_p f 


(38) 


So 


0| F- 


def 


Finally, define e '= min(ei, p|y)- F° r such an e, the right hand side of (38) is less than or equal 
to 1 if || P — Po|| f < £ ) which completes the proof. □ 
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6 Main results 


6.1 Hankel case 

First, we prove a result that generalizes Theorem [lj 
Theorem 24. For any d, n = d + 1, and r < there exists p = p(d, r) > 0 such that for all 


h d with hrank (h d ) = r and || < p in (10), the solution of © is unique and coincides with 

PI 


Proof. See Appendix |8.1| for a proof. 


□ 


Remark 25. Theorem [7] proves a special case of Theorem 24 and tells that p(d, 1) = 1 (for 
h d e 


6.2 Quasi-Hankel case 

We first prove a lemma on the limits of projectors on column spaces of quasi-Hankel matrices 


of the form (16) (for zi,...,z r in general position). 

Lemma 26. Let A = r <{ m+ ^ 1 ),and{y 1 ,...,y r } C C m be some points. Furthermore, 

assume that there exists a set V, c p C A ^ m,d °1 (for some do, 0 < do < d — 1 ) such 

that the points y i,..., y r are V -independent. 

Let P(p) denote the projector on the column space of T(a(/u/i, .. ., py r ). Then we have that 
• if r= ( m + d °) (i.e., V = then 


lim P{p) = 
o 


I r 0 
0 0 


(39) 


if r < ( m + d °) (i.e., V C A^’^j, then 


lim P{p) = 

p-t o 


Ik 

0 

0 


0 

Po 

0 


(40) 


where P 2 € C LxL , with L = ( m + d °) - K, rankjJ^} = r — K, and K ( m+ ^° 1 ). 

Proof. See Appendix [8T] for a proof. □ 

Remark 27. Generic points {y \,..., y r } with r < ( m+ ^ 1 ) satisfy conditions of Lemma 2t 
Indeed, the condition of V-independence is equivalent to 

detTr>(yi, ... ,y r ) / 0, 

which holds for generic { y\ ,..., y r } • 

By combining Lemma [23| and Lemma 26 we obtain the following theorem. 

Theorem 28. Let d" := J and r < N" := ( m ). Furthermore, let {yi, ■ ■ ■ ,y r } C 


satisfy the conditions of Lemma 26. Then there exist a constant po = po(y\■... ,y r ) > 0 such 


that for any p: 0 < p < po and points defined as zj- = pythe following holds true. 


For any nonzero coefficients c\,...,c r and the initial array defined as (15), the canonical 
completion (15) (in the problem is also the unique solution of the nuclear norm minimization 
problem ©>• 
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Proof. By Lemma 


Lemma 23 


26 


and Lemma 


19 


the matrix Pq lim p ^.o P(p) satisfies the conditions of 


Next, take e as in Lemma 23 Then, by Lemma 26, there exists po, such that 

II P{p) ~ -Poll F < £, 


which completes the proof. 


□ 


Note in the case m = 1, Theorem 28 is a weak version of Theorem 24 However, for m > 1 


in Theorem 24, it is impossible to give a uniform bound on the size of exponents (elements of 
Zk) due to fundamental issues in multivariate polynomial interpolation. 


7 Numerical results 


7.1 Hankel case 


In this case we illustrate on the examples Theorem 24 Theorem p24l only states the existence 


of such radius p, and a lower bound for p may be obtained along the lines of the proof of the 
theorem. However, this bound may be to small, and in this section we aim at showing on 
numerical experiments what would be the largest lower bound. 

All the numerical experiments are reproducible and available on request. The MATLAB 
package CVXOPT [GB14I with default settings is used for nuclear norm minimization. 

The setup for the following experiments will be similar. We take a specific hj, compute 
the solution of (|8j), and measure the Frobenius norm between the computed solution and the 


canonical completion (14). 


7.1.1 The rank-one case 


First, we consider the rank-one case. We take n = 6, and a rank-one exponential sequence hj, 
i.e. hk = A fc , where A = a + bi, a, b e (—1, l) 2 , and plot the results in Fig. [ 2 J 


Complex exponent(a+bi) 



Figure 2: Nuclear norm reconstruction (Frobenius distance), hk = A fc , A = a + bi. Black area 
correspond to the values less than 10~ 6 . 

In Fig. [2j we see that the largest lower bound for p is very close to 1. This coincides with 
the bound for real A given by Theorem [l] 
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7.1.2 The rank-two case 

We consider real sequences hj with hrank (hd) = 2. In this case the following three situations 
are possible: 

1. Ai, A 2 £ M (two simple real roots); 

2 . A 2 = A 2 0 M (two simple complex conjugate roots); 

3. Ai G R, v\ = 2 (double real root). 

The first case is considered in |DP15l Fig. 1], where it is shown (numerically) that the radius 
p is less than 1. In this section, we examine the second and the third cases. We generate the 
corresponding hd and compute the Frobenius distance between the solutions of ([ 2 ]) and ([3]), i.e., 
we compute \\S^{p) ~ ^{p)\\f- 

In Fig. [3j we plot the nuclear norm reconstructions for a 6 x 6 matrix and the last two cases. 
As seen in Fig. [3j the radius is also strictly less than 1. The radius is smaller in the case of a 
double root and also in the case when two conjugate roots are close to each other. 


Conjugate exponents 



omega 


Double real root 



Figure 3: Nuclear norm reconstruction (Frobenius distance). Top: for two complex conjugate 
roots Ai ; 2 = p ■ exp(±i7ru;) (corresponds to the damped cosine sequence ht = p l ■ cos(-7r(a; + l)t)). 
Bottom: for a double root p (corresponds to the damped linear function ht = (t ■ tan(0.75 • ir ■ 
ip) + 1) • p 1 ). Black areas correspond to the values less than 10~ 6 . 


17 

































7.2 Multiple exponents 

In this experiment, we aim at estimating the radius p based on random realizations of A&. We fix 
n = 9, and for each r € {1,..., 4} and for p 6 (0,1), we generate randomly the set {Ai,..., A r }, 
such that | Ai| = p and |Afc| < p. 

We consider two situations: 

• real roots Xk = Pk (hi this case pi = p, and pk, k > 1 are independent and uniformly 
distributed in [p; —p]) 

• complex roots A k = pke in ^ k , where pk are as in the previous example, <fik are independent, 
uniformly distributed in [0; 1], and independent of pk- 

We repeat the experiment M = 100 times, and select the maximum Frobenius error across 
all the realizations. The results are plotted in Fig. |4j 


Real roots, n = 9 




Figure 4: Nuclear norm reconstruction, n = 9, M = 400 (Frobenius distance). Top: real roots, 
bottom: complex roots. Black area correspond to the values less than 10 -5 . 


The results in Fig. [4] confirm the conclusions of Theorem 24 We see that the nuclear norm 
heuristic works up to a certain p. Note that the bounds are similar for the real and complex 


cases. 
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7.3 Quasi-Hankel case 
7.3.1 Random exponents 


In this section, we demonstrate Theorem [28} In the experiments, we fix m = 2, d = 3 and 
r = 3 (thus we are dealing with the completion of the structure ([6])). We generate random 
2/1, 2 / 2 , V3 e <C m , such that 


2/fc 


&/c T '^k 

ek + ifk ’ 


and ak,bk, ek, fk are independent identically distributed on [—0.5; 0.5]. 

For p G (0; 1], we define Zk as in Theorem 28, and consider the matrix completion with 
nuclear norm for the array ( }I5| ), where Ck are chosen to be c\ = ci = C3 = 1. We numerically 
compute the solution p of the nuclear norm minimization ([3j , and compare it with the solution 
p of the rank minimization Q (which is known from Proposition [l0]) . We repeat the experiment 
M = 10 times, and plot in Fig. [ 5 ] the dependence of ||p — p|| 2 - 



Figure 5: Nuclear norm reconstruction for quasi-Hankel matrices (parameter vector distance). 
Each of the curves represents a realization and shows the dependence of ||p — p ||2 on p. 

In Fig. [5} we see that for each realization of yk-s, there exists a radius po predicted by 
Theorem [28j 


7.3.2 Beyond the theoretical results: nonunique decompositions 

As in the previous section, we consider the case m = 2, d = 3. But we choose the rank r = 4, 
for which the conditions of Theorem 28 no longer hold true. For 3x3x3 symmetric tensors, 
this is exactly the value of generic rank [IK99j . 

The construction of the example is based on the correspondence with symmetric tensors. 
We take d linearly independent vectors in M d , i.e., 


vi,...,v d e 


Next, we consider the symmetric tensor obtained by symmetrization of a non-symmetric rank- 
one tensor: 

T = Sym(t>i (8> ■ ■ ■ (8> v d ). 
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By |CCG12j , the rank of the tensor T is equal to 2 d 1 , and the tensor admits an infinite family 
of decompositions, parameterized by 

E (- ir + - + e -® < i 6 „.«, 

£2,•••,£<2=0 


def 


where 

b E2 ,...,e d =' (71 Vi + 72 (-l) e 2 f 2 H-1- (-l) £d 7 dVd), 

and 71 ,..., 7 d G M satisfy 71 • • • 7 d = 1 . 

In order to set up the matrix completion problem, for given 71 ,, 7 ^, we first construct the 
array {h a } a£ ^m (which is equivalent to the specific tensor decomposition), where m = f d — 1 , 
as follows: 


ha — c £2 ,...,Ed^f 2 ,-,£ d ’ 

^2 j * - - j £d 0 


(41) 


where 


u £2, 


£•2 v?^d 


1 

2 d ~ 1 

A- 

1) £2+ 




^£ 2 r- 

,£d, 2 





0 £ 2 r 

,£dd 





Pe2," 

5 

2, 

■;£dd 



p£ 2,- 





Finally, we solve the nuclear norm minimization problem ([ 3 ]), for the structure defined as in (j 6 |. 
Assume for instance (d, m) = (3, 2 ) and consider 


Vi = 

~4 

1 

,V2 = 

T 

4 

,v 3 = 

T 

l 


1 


1 


4 


For this case, the nuclear norm minimization yields a 4-rank matrix completion, which corre¬ 
sponds to the minimal rank. However, the obtained completion differs from the initial completion 
(41), which indicates that the nuclear norm minimization resulted in a different decomposition 
from the infinite family of decompositions. 

We repeat the experiment for random 17 , generated as 


[vi V 2 V 3 ] = A + E, 

where the elements of E are i.i.d. uniform on [—1; 1], and A is one of the two matrices: 



'1 

1 

1 ' 


h-* 

0 

o' 

II 

CO 

1 

1 

1 

CO 

II 

0 

0 

1 

0 


1 

1 

1 


O 

0 

1 


By drawing M = 100 random realizations, we check when the Euclidean norm of the vector of 
the last singular values (< 75 ,..., 070 ) is less than 10 -4 (we say that the nuclear norm heuristics 
succeeds in this case). We observed that in the first scenario, the nuclear norm heuristic always 
failed, but for the second scenario in 77 cases the nuclear norm heuristic succeeded. 
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8 Appendix 


8.1 Proofs 

Proof of Proposition |~777[ 1. By Lemma [9j we have that 


rank Jfg(h) = r. 


Since J£g(h) is a submatrix of h) in Q, the rank of the minimal completion is at 
least r. Finally, by Lemma [9j for the canonical completion (15) we have rankJ^(h) = r. 


3. 

2 . 


In this case, the conditions of Proposition [8] take place. 


Consider a matrix of the form 


M{h) C 

C T D ’ 


(42) 


where the matrix C is known with same image as h), and the matrix D is unknown. 
Since rank^g(h) = r, for any B we have that there exists unique D such that rank(P) = r 
(for example, by [Woe89L Thm. 1.1]). 

Now, observe that for h as in ([8]), 


J£g+(h) 


'M(h) C 

C T J%b( h)J ’ 


where only J4?sb( h) is unknown (i.e., it is of the form (|42|) ). The canonical completion (15) 
has rank and therefore it is the unique completion such that rank^g+(h) = r. Finally, 
since rank Jifjg+(h) = rank=^fg(h), by Proposition [8j the overall completion is unique. 

□ 


Proof of Proposition |17[ For pc = PTZ + ipi , consider an SYD of the symmetric matrix <P(pc), 


y( Pc ) = t/ c sp c H , 


(43) 


where UqUc = V^Vc = T r , and S £ M nxn is real diagonal. Since <P(pc) is symmetric, we 
have that P'ipc) = V c^Uj is also an SVD. 

Next, we have that the matrices Be = Bn + iB x and Pc = Pr. + iPi have the form 


B n = UnVl + U X V?, B x = U X V^ - U n V^, 
Pn = UnUl + U x Uj = V n v£ + V X V?, 

Pi = U x Ul - UnUj = VnV-J - V x v£. 


Since the matrix Pc is Hermitian and Be is complex symmetric, we have that PZ = PR and 
Pj = —P Xl and Bj, = Bn and Bj = B x . 

Then from [ DHOlJ . we have that the matrix S^ ex t{Pext) admits the following SVD: 


•Pext ( Pext ) 



0 

s 




ext ? 


(44) 


where for Uc = Un + iU x and Vc = Vr + iV x 


TT def Un 

Uext - u x 


-u x 

Un 


def Vn 
Vexi ~ 


-V X 

Vn 
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We can express the matrices B ex t and P ex t (defined in (20) and (24)) for the structure 5^ ex t- 
Straightforward calculations show that 


B ext ^ U ext Vj xt = 


Btz —Bx 
Bx Btz 


p ext = U ext u] xt = v ext vj xt = 


Pn 

Pi 


-Pi 

Pn 


Next, from (29), we have that 

^ext{Pext) = 


N 


XX 

k =1 


n 


S k 0 

0 s k 


So,n ~So,i 
So,i So,n 

N 


+ 




k =1 


0 -Sk 


(45) 


where So = So.rc + iS 0 ,i- 

In what follows, we prove statements of the proposition. 


1. Analogously to (27) the subgradient of g(p ex t. ) = f \\^ext(Pext)\\* is equal to 


where 


a(M ext ) 
b(M ext ) 


U J k (^Pext) — 

b k (M ext ) = 


M ext €R 2Nx2N ,\\M ext \\ 2 < 1 


S k 0 

0 s k 

o -s k 
s k o 


■ B ex -f Q ex f A/Iexf Qext,2 ) 

/ F 


i B ex t + Q ext-M-extQ 


ext2 


def 


and for Qc = Qn + iQi = In - Pc 

Qext — 


Qn -Qi 

Qi Qn 


i Qext2 — 


Qn Qi 

-Qi Qn 


are the projectors on the left and right nullspace of ^extiPext)■ Immediately, we have that 


2 {S k ,P c )F = 
+ i 

Now let us define the matrices 

A/ext = 


S k 0 

0 s k 

o -s k 
s k o 


B, 


ext 

' F 
) B e xt 


Mi -Ms 

M 2 A^4 


, , def + M 4 

M n = -x-, 


,, def M2 + M3 def,, 

Ml = ---, M c = M n + iMx- 


(46) 


Easy calculations show that the matrix Q e xtM ext Q ex t 2 can be expressed as 

Qext JM ex t Qe X t 2 — 


Cl -c 3 

Co Cl 


(47) 
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where 


and 


Then, immediately, 


Ci=QnMiQTz+QTzMiQz—QzM2QTz+QiM±Qz, 
C2=Q'R.M2Q'iz — QTzM4Qz+QzMiQzi+QzM- i Qz, 
C3=QziM3Q'r. — QtzMiQz+QzM4Qti+QzM2Qz, 
C4=QtzM4Q k +Q k M2Qz—Qz^3QtI+QzMiQz, 


r, n* „T_Cl+C 4 , ,C 2 + C 3 
QcMcQc — -^-1-*-^-■ 


2 (Sk, Be + QcMcQc)f 

Sk 0 
0 Sk 

o -Sk 
Sk 0 


i B ext Qext-M-eoctQ 


ext 


+i 


i B e xt Qext-M-extQ 


ext 


Hence, we obtain that 

ak{M ex t) + ibk{M ex t ) = (Sk, Be + QeMeQe 


Finally, it is easy to see that for Me defined as in (|46|) 

11 -IWc 112 = 


Mtz 

-Mx 1 

[Mx 

M-ji _ 


M ext + [ °J Q ] M ext [ 5 0 J ] 


(48) 


< 11 M- e xt 112 > 

where the equality takes place if Mi = Mi and M 2 = M 3 . Hence, if ||M e;E t ||2 < 1 and 
then | Me \| 2 < 1, according to (48). Vice versa, if we are given Me = Mu + iMx with 
I|M c || 2 < 1, by taking M\ = Mi = Mn and M 2 = M 3 = Mx, we obtain M ex t with 
11 Al ex t 112 = 11 Me || 2 , which completes the proof. 


2. Let M ex t be as in 
we have that 


Q c M c Qe = ° 

which completes the proof. 


, with Mi = M 4 = M-ji, M 2 = M 3 = Mj. Then, in view of (47), 

^ Qext-M-extQext2 — 0; 


n 


Proof of Lemma \ 1 9\ (i) Denote B = 
we have that for any k G N} 


▲ ( m ’ d ') (i n this case, s < #B). Since 2 B C A, from (34), 



( 49 ) 


Therefore, Pq SkPo = 0. 

(ii) First, denote C = (in this case, n" = jfC). Then the matrix Qo has the form 


Q o 



0 

J-n—s 
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Now, as in Remark 


N 

18 we take an arbitrary H = ^Pk&k 7^ O- It is sufficient to prove that 

k =1 

the following block of the matrix QoHQq is nonzero: 

FHF t / 0, where F d = [0 /„_„»] . 

It is easy to see that the matrix FHF T has the form 

FHF T = M’ V { f), 

def 

where the values of the array f = {f a }ae2A are 


t def J 0, Ot. G A, 
Ia ~\A Pk , a = (3k, 


(3k are defined as in (33), and T> = f A\C. Finally, we have that 

2V = A (m ’ 2d) \ A (m ’ 2d " + i) D 2 A\ A, 

and therefore the matrix FHF J contains all A pk and is nonzero for any considered H. 


□ 


Proof of Theorem 24 By Lemma 23, we only need to show that for any e > 0 there exists p 
such that for all A& satisfying the statements of the theorem it holds that 


I P - -PolI f < £ ! 


where 


Pn = 


I r 0 
0 0 


G 


The distance between projectors can be expressed as 

||P - P 0 ||f = ||(I - P) - (I — P 0 )||f = 2||P 0 t/ ± || F , 

where U± is an orthonormal basis on the nullspace of P (the last equality follows from (|37[)). 
The nullspace of P coincides with the image of the matrix 

'q 0 0 . 

: qo 
q r 

0 q r 


K = 


0 


•• 0 
Qo 

0 q r _ 


G 


jinx ( n—r) 


where q(z) is the characteristic polynomial ( |10| ) (with q r = 1). Then U± G C nx ^ n ^ can be 
found from an SVD K = C/ ± SV, H , and, by submultiplicativity of matrix norms, we have that 


IP-P 


oIIf 


< ||PoiT||^||S 


— 11 | 2 
F 
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The entries of the matrix PqK are qj, for j = {0 ,... ,r — 1}. By Vieta’s formulae, these qj are 
homogeneous polynomials in the roots A*,. Therefore, there exists a universal constant C such 
that for all A&, satisfying |Afc| < 1 it holds that \qj\ < C'max(|Afc|). Hence, 


P-Pof F< C(n-r) 


max(|A fc |). 


2 mina LJ K ) 

From the well-known bounds on eigenvalues of Toeplitz matrices |GS84| . we know that 


mincr mm(^) > . min \q(z) I 2 , 

hl=i,2:ec 


which can be bounded from below by a constant if the roots Xk satisfy |A*,| < po for 0 < po < 1. 
Hence, there exists a constant C such that for all |Afc| < po the following inequality holds true 

||P - Pq\\f < C max(|Afc|). 


d 


Proof of Lemma 26_. Denote M = ffA and assume that V = {aq,..., ct r } and 5T> = {/3q,..., (3j}. 
Since y\.... ,y r are D-independent, there exist coefficients [wi,j\ r fj = \ such that the polynomials 


Pj{y) l = '(/' ~^2 w i,jy ai 

i =1 

vanish on {yi,... ,y r }- The polynomials Pj(y) constitute a so-called border basis [ Ste04, Ch. 2] 
of the interpolating ideal of {y\, ..., y r } (the ideal of polynomials vanishing at {t/i,..., y r })- 
Now, define the following polynomials: 


Pj, p (y) d = y fll Kl , 

2—1 

which for p > 0 are equal to p^^Pj{p~ l y) and therefore vanish {pyi, ..., py r }- Moreover, Pj, p (y ) 
constitute a border basis of the interpolating ideal of {pyi, ■ ■ ■, py r }- 

Now let us build the left kernel /C p c = l.ker. {pAipyii ■ ■ ■ > PVr )) of the corresponding Vander¬ 
monde matrix. In what follows, we construct a basis of IC p . Consider sets of multiindices 

£i = {ei,i, ■ ■ ■, ei,m}, • • • ,£j = {ej,i, • ■ ■, ej,nj}, 
such that m + ... + nj = M — r and 

(£\ + {/ 3 i}) U • • • U {£m + {/ 3 m }) = A \ T>. 

Then we have that the polynomials {qj.i. P }'j= \ i=1 defined as 

Qj,i, P {y) = f y ej,i PjAy ) 

are linearly independent. Finally, the monomials of each polynomial qj P)P (y) have degree at 
most d, and therefore, there exist vectors qj t i tP € C AI such that 

Qj,i, P i.y) = [y ai • • • y aM ] 
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Since all qi,j, P (y ) vanish on y \,..., y r , and qij are linearly independent, we have that 

Kp = colspan {qj,i, P }j=l i=1 - 

Now we calculate the limit of IC P as p —> 1. Consider the cases from the statement of the lemma. 

(i) If r = ( m ^ rf °), we have that V = and 5V = A( m ’ do+1 ). Hence, \/3\ > |a| for all 

(3 S ST>, a G T>, and we have that 

= Qj,i, o(y) = y Sj ’ i+l3j - 

p—>0 

Since all sums + /3j are distinct, we have that 


lim AA = colspan 

p-S-0 



which implies (39). 

(ii) Let r 
ordered such that 


Without loss of generality, we assume that elements of T> and 5T> are 


k (m,do-D = C, = {a K+1 , ..., a r } C 

C 2 = {(3i, ... ,/3 r } C C 3 = {/3 T+ i, • • • , A/} C A^° +1 ), 


and T = L + K — r, V = ^ m ’ do U Ci, SV = C 2 U C 3 . Note that since C 3 = bV \ A ^ m,d °' ) , we 
have that C 3 = <5Ci. Now we have that 


lin i<2W(2/) = Qj,i,o(y) 

p — 


y e j,i+@j ^ 


r 


£ wijy ai ), 

l=K +1 


.7 < T, 
3 > T. 


Finally, note that we can order eyj in such a way that ey i = 0 for all j. Taking into account 
that bC i = C 3 , we can cancel lower monomials in qj } i t o(y), i > 1 without changing the linear 
space spanned by gyyo- More precisely, the set of polynomials {qi,i} f=\ l= \, defined as 

~ / v = f ^,i,o(y), (t = 1) and (j < T), 

|y £ j,*+/ 3 j ) (7 > 1) or (j > T), 

spans the same subspace as l= \ ■ Therefore, the limit of the kernel has the form 


* 0 

lim Kp = colspan It 0 

p —‘>0 T 

1 N-(K+L)_ 


where * denotes the elements of qijfi(y) for j = 1 ( i.e ., the first T columns correspond to gyj(y) 
for i=l and j <T). Hence, the projector on the orthogonal complement of limp_;.o^cp has the 
form (40). O 
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